Applicant: LIPTONetal. 
Appl. No. 10/667,148 

In the Claims: 

The following listing of claims will replace all prior versions and/or listings of claims in 
the application: 
Listing of Claims: 

1 . (Previously Presented) A method of implementing an intelligent video surveillance 
system, comprising: 

obtaining a frame sequence from an input video stream; 

executing a first-pass method for each frame of the frame sequence, the first-pass method 
comprising the steps of: 

aligning the frame with a scene model; and 
updating a background statistical model; 
finalizing the background statistical model; 

executing a second-pass method for each frame of the frame sequence, the second-pass 
method comprising the steps of: 

labeling each region of the frame; and 

performing spatial/temporal filtering of the regions of the frame; 
identifying and classifying objects using the labeled and filtered regions; and 
analyzing behaviors of at least one of the objects. 

2. (Original) A computer-readable medium comprising software implementing the method of 
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Claim 1. 

3. (Original) An intelligent video surveillance system comprising a computer system 
comprising: 

a computer; and 

a computer-readable medium according to Claim 2. 

4. (Original) A method of implementing an intelligent video surveillance system, comprising: 

obtaining a frame sequence from a video stream; 

for each frame in the frame sequence, performing the following steps: 

aligning the frame with a scene model; 

building a background statistical model; 

labeling the regions of the frame; and 

performing spatial/temporal filtering; 
identifying and classifying objects based on the results of the labeling and filtering; and 
analyzing behaviors of at least one object. 

5. (Original) A computer-readable medium comprising software implementing the method of 
Claim 4. 
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6. (Original) An intelligent video surveillance system comprising a computer system 
comprising: 

a computer; and 

a computer-readable medium according to Claim 5. 



7. (Original) A method of implementing an intelligent video surveillance system, comprising: 
obtaining a frame sequence from a video stream; 
for each frame in the frame sequence, performing the following steps: 
aligning the frame with a scene model; 

building a background statistical model and a secondary statistical model; 

labeling the regions of the frame; and 

performing spatial/temporal filtering; 
identifying and classifying objects based on the results of the labeling and filtering; and 
analyzing behaviors of at least one object. 



8. (Original) A computer-readable medium comprising software implementing the method of 
Claim 7. 



9. (Original) An intelligent video surveillance system comprising a computer system 
comprising: 



-5- 



Applicant: LIPTONetal. 
Appl.No. 10/667,148 



a computer; and 

a computer-readable medium according to Claim 8. . 

10. (Original) A method of implementing an intelligent video surveillance system, comprising: 

segmenting video into foreground and background components, the segmenting 
comprising: 

obtaining a sequence of video frames; 

building and updating at least one background statistical model for each region of 
the video frames, based on the video frames; and 

assigning labels to the regions, based on the at least one background statistical 

model; 

identifying and classifying objects based on the labeled regions; and 
analyzing behaviors of at least one object. 

1 1 . (Original) A computer-readable medium comprising software implementing the method of 
Claim 10. 

12. (Original) An intelligent video surveillance system comprising a computer system 
comprising: 

a computer; and 
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a computer-readable medium according to Claim 1 1 . 

13. (Previously Presented) The method of Claim 1 , wherein said analyzing behaviors of at 
least one of the objects comprises: 

tracking at least one of the objects. 

14. (Previously Presented) The method of Claim 1, further comprising: 

creating at least one rule to detect at least one specific activity; 
wherein said analyzing behaviors of at least one of the objects includes applying the at 
least one rule. 

15. (Previously Presented) The method of Claim 14, wherein said at least one rule includes at 
least one virtual tripwire and determining when the at least one virtual tripwire is crossed. 

16. (Previously Presented) The method of Claim 14, wherein said at least one rule includes a 
definition of at least one area and the determining at least one of when an object enters, when an 
object leaves, and when an object loiters in the at least one area. 
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17. (Previously Presented) The method of Claim 14, wherein said at least one rule includes at 
least one of determining when an object is added to a scene and determining when an object is 
removed from a scene. 

18. (Previously Presented) A method of implementing an automated closed-circuit television 
(CCTV) surveillance system, comprising: 

providing CCTV equipment generating an input video stream; and 
implementing the method of Claim 1 . 

19. (Previously Presented) A method of implementing an automated security system, 
comprising the method of Claim 1 . 

20. (Previously Presented) A method of implementing an automated anti-terrorism system, 
comprising the method of Claim 1 . 

2 1 . (Previously Presented) A method of implementing an automated market research system, 
comprising the method of Claim 1 . 

22. (Previously Presented) The method of Claim 21, wherein said analyzing behaviors of at 
least one of the objects comprises: 
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tracking behaviors of at least one of the objects in at least one retail location. 

23. (Previously Presented) A method of implementing an automated traffic monitoring 
system, comprising the method of Claim 1. 

24. (Previously Presented) The method of Claim 23, wherein said analyzing behaviors of at 
least one of the objects comprises at least one of: 

detecting wrong-way traffic; 
detecting a broken-down vehicle; 
detecting an accident; and 
detecting a road blockage. 

25. (Previously Presented) A method of implementing a video compression system 
comprising the method of Claim 1 . 

26. (Previously Presented) The method of Claim 4, wherein said analyzing behaviors of at 
least one of the objects comprises: 

tracking at least one of the objects. 

27. (Previously Presented) The method of Claim 4, further comprising: 
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creating at least one rule to detect at least one specific activity; 
wherein said analyzing behaviors of at least one of the objects includes applying the at 
least one rule. 

28. (Previously Presented) The method of Claim 27, wherein said at least one rule includes at 
least one virtual tripwire and determining when the at least one virtual tripwire is crossed. 

29. (Previously Presented) The method of Claim 27, wherein said at least one rule includes a 
definition of at least one area and the determining at least one of when an object enters, when an 
object leaves, and when an object loiters in the at least one area. 

30. (Previously Presented) The method of Claim 27, wherein said at least one rule includes at 
least one of determining when an object is added to a scene and determining when an object is 
removed from a scene. 

3 1 . (Previously Presented) A method of implementing an automated closed-circuit television 
(CCTV) surveillance system, comprising: 

providing CCTV equipment generating an input video stream; and 
implementing the method of Claim 4. 
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32. (Previously Presented) A method of implementing an automated security system, 
comprising the method of Claim 4. 

33. (Previously Presented) A method of implementing an automated anti-terrorism system, 
comprising the method of Claim 4. 

34. (Previously Presented) A method of implementing an automated market research system, 
comprising the method of Claim 4. 

35. (Previously Presented) The method of Claim 34, wherein said analyzing behaviors of at 
least one of the objects comprises: 

tracking behaviors of at least one of the objects in at least one retail location. 

36. (Previously Presented) A method of implementing an automated traffic monitoring 
system, comprising the method of Claim 4. 

37. (Previously Presented) The method of Claim 36, wherein said analyzing behaviors of at 
least one of the objects comprises at least one of: 

detecting wrong- way traffic; 
detecting a broken-down vehicle; 
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detecting an accident; and 
detecting a road blockage. 

38. (Previously Presented) A method of implementing a video compression system, 
comprising the method of Claim 4. 

39. (Previously Presented) The method of Claim 7, wherein said analyzing behaviors of at 
least one of the objects comprises: 

tracking at least one of the objects. 

40. (Previously Presented) The method of Claim 7, further comprising: 

creating at least one rule to detect at least one specific activity; 
wherein said analyzing behaviors of at least one of the objects includes applying the at 
least one rule. 

41 . (Previously Presented) The method of Claim 40, wherein said at least one rule includes at 
least one virtual tripwire and determining when the at least one virtual tripwire is crossed. 
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42. (Previously Presented) The method of Claim 40, wherein said at least one rule includes a 
definition of at least one area and the determining at least one of when an object enters, when an 
object leaves, and when an object loiters in the at least one area. 

43. (Previously Presented) The method of Claim 40, wherein said at least one rule includes at 
least one of determining when an object is added to a scene and determining when an object is 
removed from a scene. 

44. (Previously Presented) A method of implementing an automated closed-circuit television 
(CCTV) surveillance system, comprising: 

providing CCTV equipment generating an input video stream; and 
implementing the method of Claim 7. 

45. (Previously Presented) A method of implementing an automated security system, 
comprising the method of Claim 7. 

46. (Previously Presented) A method of implementing an automated anti-terrorism system, 
comprising the method of Claim 7. 
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47. (Previously Presented) A method of implementing an automated market research system, 
comprising the method of Claim 7. 

48. (Previously Presented) The method of Claim 47, wherein said analyzing behaviors of at 
least one of the objects comprises: 

tracking behaviors of at least one of the objects in at least one retail location. 

49. (Previously Presented) A method of implementing an automated traffic monitoring 
system, comprising the method of Claim 7. 

50. (Previously Presented) The method of Claim 49, wherein said analyzing behaviors of at 
least one of the objects comprises at least one of: 

detecting wrong-way traffic; 
detecting a broken-down vehicle; 
detecting an accident; and 
detecting a road blockage. 

5 1 . (Previously Presented) A method of implementing a video compression system, 
comprising the method of Claim 7. 
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52. (Previously Presented) The method of Claim 10, wherein said analyzing behaviors of at 
least one of the objects comprises: 

tracking at least one of the objects. 

53. (Previously Presented) The method of Claim 10, further comprising: 

creating at least one rule to detect at least one specific activity; 
wherein said analyzing behaviors of at least one of the objects includes applying the at 
least one rule. 

54. (Previously Presented) The method of Claim 53, wherein said at least one rule includes at 
least one virtual tripwire and determining when the at least one virtual tripwire is crossed. 

55. (Previously Presented) The method of Claim 53, wherein said at least one rule includes a 
definition of at least one area and the determining at least one of when an object enters, when an 
object leaves, and when an object loiters in the at least one area. 

56. (Previously Presented) The method of Claim 53, wherein said at least one rule includes at 
least one of determining when an object is added to a scene and determining when an object is 
removed from a scene. 
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57. (Previously Presented) A method of implementing an automated closed-circuit television 
(CCTV) surveillance system, comprising: 

providing CCTV equipment generating an input video stream; and 
implementing the method of Claim 10. 

58. (Previously Presented) A method of implementing an automated security system, 
comprising the method of Claim 10. 

59. (Previously Presented) A method of implementing an automated anti-terrorism system, 
comprising the method of Claim 10. 

60. (Previously Presented) A method of implementing an automated market research system, 
comprising the method of Claim 10. 

61 . (Previously Presented) The method of Claim 60, wherein said analyzing behaviors of at 
least one of the objects comprises: 

tracking behaviors of at least one of the objects in at least one retail location. 

62. (Previously Presented) A method of implementing an automated traffic monitoring 
system, comprising the method of Claim 10. 
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63. (Previously Presented) The method of Claim 62, wherein said analyzing behaviors of at 
least one of the objects comprises at least one of: 

detecting wrong- way traffic; 
detecting a broken-down vehicle; 
detecting an accident; and 
detecting a road blockage. 

64. (Previously Presented) A method of implementing a video compression system, 
comprising the method of Claim 4. 

65. (Previously Presented) An apparatus for intelligent video surveillance adapted to perform 
the method comprising: 

obtaining a frame sequence from an input video stream; 

executing a first-pass method for each frame of the frame sequence, the first-pass method 
comprising the steps of: 

aligning the frame with a scene model; and 

updating a background statistical model; 
finalizing the background statistical model; 

executing a second-pass method for each frame of the frame sequence, the second-pass 



-17- 



Applicant: LIPTONetal. 
Appl.No. 10/667,148 



method comprising the steps of: 

labeling each region of the frame; and 

performing spatial/temporal filtering of the regions of the frame; 
identifying and classifying objects using the labeled and filtered regions; and 
analyzing behaviors of at least one of the objects. 



66. (Previously Presented) The apparatus of claim 65 wherein the apparatus comprises 
application-specific hardware to emulate a computer and/or software adapted to perform said 
obtaining, said executing a first-path method, said finalizing, said executing a second-path 
method, said identifying, and said analyzing. 



67. (Previously Presented) An apparatus for intelligent video surveillance adapted to perform 
the method comprising: 

obtaining a frame sequence from a video stream; 
for each frame in the frame sequence, performing the following steps: 
aligning the frame with a scene model; 
building a background statistical model; 
labeling the regions of the frame; and 
performing spatial/temporal filtering; 
identifying and classifying objects based on the results of the labeling and filtering; and 
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analyzing behaviors of at least one object. 

68. (Previously Presented) The apparatus of claim 67 wherein the apparatus comprises 
application-specific hardware to emulate a computer and/or software adapted to perform said 
obtaining, said aligning, said building, said labeling, said filtering, said identifying, and said 
analyzing. 

69. (Previously Presented) An apparatus for intelligent video surveillance adapted to perform 
the method comprising: 

segmenting video into foreground and background components, the segmenting 
comprising: 

obtaining a sequence of video frames; 

building and updating at least one background statistical model for each region of 
the video frames, based on the video frames; and 

assigning labels to the regions, based on the at least one background statistical 

model; 

identifying and classifying objects based on the labeled regions; and 
analyzing behaviors of at least one object. 
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70. (Previously Presented) The apparatus of claim 69 wherein the apparatus comprises 
application-specific hardware to emulate a computer and/or software adapted to perform said 
segmenting, said obtaining, said building, said assigning, said identifying, and said analyzing. 

71 . (New) A method of implementing an intelligent video surveillance system, comprising: 

building and updating at least one background statistical model for an input video 
sequence; 

labeling regions of frames of the input video sequence based on the at least one 
background statistical model; 

identifying and classifying at least one object in the input video sequence based on the 
labeled regions; and 

analyzing behavior of said at least one object. 

72. (New) A computer-readable medium comprising software implementing the method of 
Claim 71. 

73. (New) An apparatus for intelligent video surveillance adapted to perform the method 
comprising: 

building and updating at least one background statistical model for an input video 
sequence; 

labeling regions of frames of the input video sequence based on the at least one 
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background statistical model; 

identifying and classifying at least one object in the input video sequence based on the 
labeled regions; and 

analyzing behavior of said at least one object. 

74. (New) The apparatus of claim 73 wherein the apparatus comprises application-specific 
hardware to emulate a computer and/or software adapted to perform said segmenting, said 
obtaining, said building, said assigning, said identifying, and said analyzing. 
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